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Fluorescence microscopy is widely used for the study of biological specimens. Decon¬ 
volution can significantly improve the resolution and contrast of images produced using 
fiuorescence microscopy; in particular, Bayesian-based methods have become very popu¬ 
lar in deconvolution fiuorescence microscopy. An ongoing challenge with Bayesian-based 
methods is in dealing with the presence of noise in low SNR imaging conditions. In this 
study, we present a Bayesian-based method for performing deconvolution using dynami¬ 
cally updated nonparametric nonstationary expectation estimates that can improve the 
fiuorescence microscopy image quality in the presence of noise, without explicit use of 
spatial regularization. 


Introduction 


Fluorescence microscopy has become an essential tool for visualizing cellular structures and properties 
of biological specimens by using fluorescence as a means of identifying cellular components. For 
example, using fluorescent live/dead stains as well as fluorescently-labeled particles or molecules 
(such as antibodies), researchers are able to characterize the effect of a treatment on cell population. 
While many techniques allow for a quantitative assessment of cell phenotype, in the case of low 
cell numbers and when studying structure and/or viability of adherent cells, fluorescence microscopy 
remains the tool of choice. Features of the biological sample and staining procedures may affect the 
quality of the acquired image. Deconvolution fluorescence microscopy provides a way of overcoming 
the inherent limitations of the optical microscopy system by making use of the intrinsic properties of 
the system to increase image contrast and resolution after image acquisition. This allows for improved 
visualization at subresolution scales. 

A number of deconvolution methods have been previously introduced, and include nearest neighbor 
methods (^, Wiener Altering [^, iterative constrained regularization methods [2]-[^, and Bayesian- 
based methods (7|- [^[26] - |28] . Bayesian-based deconvolution methods are statistical methods that 
model images, point spread functions (PSF), and imaging noise as probability distributions, and 
can be separated into two classes: maximum likelihood (ML) methods, and maximum a posteriori 
(MAP) methods. While current Bayesian-based deconvolution methods account for inherent image 
noise characteristics, they are sensitive to low signal-to-noise ratio (SNR) imaging conditions where 
prior model parameter estimation is very challenging due to the presence of noise, thus leading to 
reduced deconvolved image quality. A lot of focus on deconvolution fluorescence microscopy in dealing 
with the presence of noise in low SNR imaging conditions has revolved around the introduction of 


spatial regularization priors 17 ,^ to impose spatial smoothness for suppressing noise characteristics. 
Other strategies, such as ForWarD (^ , have been proposed to deal with the presence of noise by 
first performing deconvolution followed by noise suppression 22-^. In this study, we investigate 


and attempt to mitigate this issue from a different perspective by introducing an MAP method that 
performs deconvolution on fluorescence microscopy images using dynamically updated nonparametric 
nonstationary expectation estimates, which does not make explicit use of spatial regularization. 


1 






Here, we model the deconvolved fluorescence microscopy image (/) and the measured fluorescence 
microscopy image {g) as probability distributions. In the proposed MAP method, the goal is to 
determine the most probable deconvolved image / given the measured image g, based on prior 
knowledge related to /. The MAP problem can be formulated as: 


/ = argmax^ p(/|^) , (1) 

where p{f\g) is the conditional probability of / given g. We model g as Poisson distributed, and 
model / as a nonstationary process with a nonstationary expectation E{fs) and a variance r^, where s 
denotes the pixel site location. In previous MAP methods j^, the measured fluorescence microscopy 
image g was used as an estimate of E{fs), based on the assumption that the measured image, having 
undergone the influence of the PSF, is similar to a nonstationary local average and thus representative 
of a nonstationary expectation E{fs). However, this assumption may not be reliable in high noise 
situations under low SNR imaging conditions, as well as result in high-varying estimates depending 
on the inherent PSF. One can obtain a more reliable estimate of E{fg) by computing local windowed 
averages of g] however, this method is also highly sensitive to the presence of high noise levels. To 
tackle this issue, we introduce a dynamically updated nonparametric kernel-based estimate (based 
on the work by 29,^) of E(fs) that updates with each iteration. Hence, we derive the iterative 


solution of the proposed MAP method (see Methods for full derivation), yielding: 




n 


H—s gs 


( 2 ) 


where /| denotes the deconvolved image at iteration j, denotes convolution operator, H denotes 
the point spread function (PSF), A and (3 denote relaxation parameters, and Wi denotes a window 
centered at location i. 

Eq. ([^ denotes an update step of the iteration solution. The importance of introducing a dy¬ 
namically updated nonparametric kernel-based estimate of E{fs) that updates with each iteration is 
that it allows for a reliable estimate of E{fs) under the influence of image noise. Hence, the proposed 
MAP method results in a solution that remains stable under low SNR imaging conditions, leading 
to improved image quality of the deconvolved image /. The proposed solution, to the best of our 
knowledge, is the first to consider to incorporate dynamically updated nonparametric nonstation¬ 
ary expectation estimates within a Bayesian theoretical framework for the purpose of deconvolution 
fluorescence microscopy, which allows for more stable solutions without the explicit use of spatial 
regularization. 


Results 

To evaluate the proposed MAP method with dynamically updated nonparametric nonstationary ex¬ 
pectation estimates (which we will refer to as MAP-D in figures), we first simulated a fluorescence mi¬ 
croscopy data set with fluorescence-stained cell populations using a modified version of SIMCEP [3l| , 
a computational framework for simulating fluorescence microscopy images of fluorescence-stained cell 
populations (see Fig. lb and Methods). For comparison purposes, three other Bayesian-based meth¬ 
ods were tested: Lucy-Richardson (LR) deconvolution j7[[^, Markov-Chain Monte-Carlo Wiener-Hunt 
(MCMC-WH) deconvolution [^, and Hunt MAP (MAP-Hunt) deconvolution method with Poisson 
likelihood and Gaussian image prior [^. Note that LR, MAP-Hunt, and the proposed MAP-D 
methods are performed without the explicit use of spatial regularization to test the hypothesis that 
MAP-D can achieve improved fluorescence microscopy image quality in the presence of noise, without 
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explicit use of spatial regularization. The proposed MAP-D method (Fig. If) was able to achieve a 
significant increase in contrast and resolution when compared to the original fluorescence microscopy 
acquisition, as well as a noticeable SNR increase (Figs. If). This is most evident by the fact that all 4 
subcellular structures inside the cytoplasm of each cell are visible in the deconvolved image produced 
using the proposed MAP-D method when compared to the original fluorescence microscopy acquisi¬ 
tion. Compared to MAP-Hunt deconvolution |[^ (Fig. le), the proposed MAP-D method achieved 
greater increases in contrast and resolution, and noticeable SNR improvements. This is particular 
relevant when studying bacteria binding to cells as the increase in contrast and resolution enables to 
accurately identify individual particles. The proposed MAP-D method also provides added benefits 
when studying cell processes such as microvesicles and cell interactions with nanoparticles. 

The proposed MAP-D method was then evaluated on fluorescence microscopy acquisitions of cells 
obtained from an ocular surface wash following sodium fluorescein instillation (see Methods). Fig. 
2a shows an acquisition of cell aggregates, some staining with fluorescein (green) while others stain red 
with the dead stain (ethidium bromide). Fig. 2f shows a live/dead (calcein blue/ethidium bromide- 
red) acquisition of two corneal epithelial cells. (Figs. 2a and 2f and Methods). The proposed 
MAP-D method (Fig. 2e and 2j) was able to achieve a significant increase in contrast and resolution 
when compared to the original fluorescence microscopy acquisitions (Figs. 2a and 2f). Compared 
to MAP-Hunt deconvolution (Fig. 2d and 2i), the proposed MAP-D method achieved greater 
increases in contrast and resolution, and moderate SNR improvements. Based on a cross-sectional 
prohle of an area of interest (Fig. 2k) and a background area (Fig. 21) from Fig. 2a, we can 
observed that the proposed method improve contrast and resolution without introducing undesirable 
artifacts. Additional deconvolution results from microscopy acquisitions in Fig. 3 further reinforce 
these observations. In the original fluorescence microscopy acquisitions (Figs. 3a, 3f, 3k and 3p), due 
to low contrast levels, it was difficult to determine the nature of the cells present as the cell walls 
could not be clearly identified. Deconvolution results achieved using LR, MAP-Hunt, and MCMC- 
WH resulted in a cell wall that is easier to detect but also introduced noticeable noise artifacts and 
thus reduced the overall image quality (Figs. 3b-3d, 3g-3i, 31-3n and 3q-3s). With the proposed 
MAP-D method, improved cell wall identihcation as well as improved contrast were obtained without 
increasing noise artifacts (Figs. 3f, 3k, 3o, 3t). 


Discussion 

In this study, we introduce a Bayesian-based method for performing deconvolution using dynami¬ 
cally updated nonparametric nonstationary expectation estimates that can improve the fluorescence 
microscopy image quality. The key factor to the improved performance of the proposed method com¬ 
pared to the other tested Bayesian-based deconvolution methods is the introduction of dynamically 
updated nonparametric expectation estimates into a Bayesian-based deconvolution framework, which 
greatly increases the stability of the solution in the presence of noise, without explicit use of spatial 
regularization. Furthermore, by dynamically updating the expectation estimate at each iteration, the 
estimate of E{fs) becomes increasingly more reliable as the deconvoluted image estimate fs becomes 
more accurate. The only free parameters of the proposed MAP method are A, /?, W, and the number 
of iterations for deconvolution, which can be adjusted by the user to hnd a tradeoff between image 
quality and computational costs. 

Analysis of ex vivo collected human cells present different challenges. In the context of our study, 
due to the nature of the collection (gentle eye wash), the presence of debris can make cell identification 
difficult (as illustrated in Fig. 2a and 3a). Corneal epithelial cells collected from the ocular surface 
can also uptake various stains during the procedures; green from instillation of sodium fluorescein on 
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the ocular surface, blue from calcein blue if the cell is alive and red from ethidium bromide if the 
cell is dead. Ghost cells, cells without a nucleus, may also be present in the ocular wash. Current 
research aims at determining how corneal epithelial cells uptake fluorescein and how treatment, such 
as different combinations of lens cleaning solutions and contact lenses, can affect the ocular surface 
and cell shedding (the cells collected during the gentle eye wash). Fluorescence microscopy allows 
to characterize these cells, however poor image quality can prevent efficient counting and investiga¬ 
tion of the population of cells present in the ocular wash. While confocal microscopy may be seen 
as an alternative, cost, cell transport, acquisition time and operator experience make confocal mi¬ 
croscopy ill-suited for such investigations. Other techniques have been used to collect and study these 
cells (33|[M) ; however, the image quality of fluorescein-stained cells was also a challenge, highlighting 
the need of a back-end approach to help resolve such issues as the cell collection technique is not able 
to provide significant improvements in the image quality of the cell samples. This is likely due to the 
complexity of ex vivo cell collection, and the fragility of the cells studied which restricts processing 
protocols and times. The ability to increase fluorescence image quality using the proposed MAP-D 
method so that cellular membranes and staining patterns can be better identified can further enhance 
the use and benefits of fluorescence microscopy. Furthermore, increasing fluorescence image quality 
using the proposed MAP-D method could lead to improvements in tasks used for data analysis such 
as cell counting [35] and data alignment jsM 37 . As such, the proposed method can be valuable for 


improving fluorescence microscopy image quality to facilitate for the study of biological specimens, 
more specifically ex vivo collected samples where the presence of debris or other collection artifacts 
may exist and where significant cell processing is not an option due to low cell numbers and poor 
cell viability. Future work involves the investigation of alternative nonparametric expectation estima¬ 
tion approaches, such as Monte Carlo estimation strategies 38 10. to further improve fluorescence 
microscopy image quality in the presence of noise. 


Methods 


Derivations. 

A full derivation of the iterative solution of the proposed MAP method can be described as follows. 
Let S' be a set of pixel locations into a discrete lattice C and s G S' be a pixel location in C. Let 
F = {Fs\s G S'} and G = {Gs|s G S'} be random fields on S', with Fg and Gg taking on values 
representing the deconvolved fluorescence microscopy image and measured fluorescence microscopy 
image at pixel s respectively. Let / = {fg\s G S'} and g = {s'sis G S'} be realizations of F and G 
respectively. Let H denote the point-spread function. The goal is to estimate / given g, based on 
prior knowledge related to /. 


this is equivalent to: 


/ = aigmaxj: p{f\g) , 


( 3 ) 


/ = argmax^ p{9\f) p{f), (4) 

where p{g\f) is the likelihood and p{f) is the prior. In the case of fluorescence microscopy, image 
noise is primarily related to quantum photon emission and as such can express p{g\f) as follows: 


piaif)= n 

seS 


9 s'- 


( 5 ) 
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where denotes the convolution operator. As a prior model p{f), we employ a Gaussian prior model 
with a nonstationary expectation E{fs) and a variance 


p(/)=n'' 

seS 


( 6 ) 


Therefore, Eq. (|^ can be reformulated 


as: 


/ TT _(fs-E(Mp 

j = argmaxj I M -^-e 272 






(7) 


This is equivalent to minimizing the negative logarithm of p {g\f)p{f), given us: 

1 


{h 0 /,) - g, log (j/ 0 /.) + log (g,!) + ^ (/, - £(/.))= 


/ = argmiuy 

To hud the solution that minimizes Eq. (|8|), we take the derivative and set it to zero: 

9s 


ds. 


( 8 ) 
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where A is a relaxation parameter. Simplifying further: 

'H_s(S)9. 


+ A(/,-E(/,)) = 0 
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?/ +1 . 

To establish the iterative solution, let us assume that = 1 at convergence, where denotes 

Js 


the deconvolved image at iteration j. Therefore, 




+ A E(/,)-/i = 
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Since E{fs) is unknown, we instead use an estimate E{fs), giving us: 

H—s ( 81 9s 


( 12 ) 

(13) 


fi+i _ fj 
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(14) 


Contrary to existing methods 0. which employ the measured fluorescence microscopy image g 
as an estimate of E{fs), we introduce a dynamically updated nonparametric kernel-based estimate 
of E{fs) that updates with each iteration for a more reliable estimate of E{fs) under the influence of 
noise, particularly low-SNR scenarios, which can be expressed by: 


Hfs) = 


- fr)fi 


(15) 
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where Wg denotes a window centered at location s and is a kernel function. Plugging Eq. (15) into 
Eq. (14), with the following Gaussian kernel function: 


- fs) = (16) 

where /3 is a relaxation parameter, gives us the final iterative solution of the proposed MAP method, 
yielding: 
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(17) 


Imaging apparatus. 

For this study, the Nikon fluorescence microscope Eclipse, TE2000-S, with a triple band fluorescence 
hlter module (DAPI/FITC/TRITC) was used (Fig. 3k) to obtain fluorescence microscopy acqui¬ 
sitions of cells obtained from an ocular surface wash following sodium fluorescein instillation. The 
microscope was equipped with a motorized XY stage and a microcontroller from Applied Scientific 
Instrument. A DS-Fil digital camera and its controller as well as the NIS-Elements software for 
multi-dimensional acquisition were purchased from Nikon. Image acquisition was performed using a 
20 x objective (resulting in a 200x magnihcation when accounting for magnification of the eye piece). 
Before deconvolution on fluorescence microscopy imaging data can take place with the proposed MAP 
method, it is first necessary to estimate the PSF of the microscope. In this study, the Nikon fluo¬ 
rescence microscope Eclipse TE2000-S was characterized based on a number of image acquisitions of 
point source targets to estimate the PSF of the microscope. In the situation where the PSF of the 
microscope is not known, PSFs obtained using simulations or estimated using other means may also 
be used in the proposed MAP deconvolution method. 


Simulation of fluorescence microscopy data sets. 

To evaluate the proposed MAP method, we simulated a ground-truth fluorescence microscopy data 
set with fluorescence-stained cell populations (Fig. la) and a convolved and noisy fluorescence mi¬ 
croscopy data set from the same cell population (Fig. lb) using a modified version of SIMCEP (3l| , 
a computational framework for simulating fluorescence microscopy images of fluorescence-stained cell 
populations. The only modification to SIMCEP is the introduction of an additional Poisson process 
to generate pixel intensities in the SIMCEP framework. We conflgure SIMCEP to generate cell pop¬ 
ulations with visible cytoplasm, nuclei, and a total of 4 subcellular structures inside the cytoplasm 
of each cell, and allowing for overlapping cells, with energy of autofluorescence and variance of CCD 
detector noise at 0.05 and 0.001, respectively. 


Sample preparation. 

Sample preparation was performed for this study in the following manner. Following sodium fluores¬ 
cein instillation, cells from the ocular surface were collected by a gentle eye wash using the Ocular 
Surface Cell Collection Apparatus |^. Cells were centrifuged and then stained using the live/dead 
stain (calcein blue/ethidium bromide). Following a 45 minute incubation in the dark at room tem¬ 
perature, cells were imaged immediately using a Nikon Eclipse fluorescence microscope. Some cells 
uptook sodium fluorescein and appeared green; live cells stained with Calcein blue and their nucleus 
appeared blue while dead cells stained with ethidium bromide and were characterized by a red nucleus. 

Implementation details. 

The proposed MAP method (referred to here as MAP-D) is implemented in MATLAB (The Math- 
Works, Inc.), with the nonparametric kernel-based estimate of E{f{s)) implemented in C++ and 
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compiled a dynamically linked MATLAB Executable (MEX) for to improve computational speed. 
The only free parameters of the proposed MAP-D method are A, /?, W, and the number of iterations 
for deconvolution, which can be adjusted by the user to find a tradeoff between image quality and 
computational costs. For the experiments using fluorescence microscopy acquisitions, the parameters 
A, /3, W, and the number of iterations for deconvolution for MAP-D are set to 0.2, 625, a 81-sample 
window, and 50, respectively. For Lucy-Richardson (LR) deconvolution [^|^, the only free parameter 
was the number of iterations, and that was set to 50. For the Hunt MAP (MAP-Hunt) deconvolu¬ 
tion method with Poisson likelihood and Gaussian image prior [lO) , the free parameters are A and 
the number of iterations, and these were set to 0.2 and 50, respectively. All parameters are chosen 
to be consistent across the LR, MAP-Hunt, and MAP-D methods (e.g., all shared parameters are 
the same) and provide strong image contrast and resolution improvements across tested acquisitions. 
Finally, for Markov-Chain Monte-Carlo Wiener-Hunt (MCMC-WH) deconvolution 12 , the code was 
provided in MATLAB by the authors of 12 , and the free parameters are the number of iterations 


for burn-in and the maximum number of iterations, and these were set to 30 and 150, respectively, 
based on author-provided information. The PSF of the microscope obtained as discussed above is 
used as an input parameter for all tested deconvolution methods. For these configurations, the im¬ 
plementations of MAP-D, LR, MAP-Hunt, and MCMC-WH can process 800 x 800 three-channel 
fluorescence microscopy acquisitions in ~45s, ~39s, ~40s, and ~79s, respectively, on an Intel(R) 
Core(TM) i5-3317U CPU at 1.70GHz CPU with 4Gb RAM. 
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Figure Legends 



(b) (e) (f) 


Figure 1: Application to simulated fluorescence microscopy data of fluorescence-stained cell popula¬ 
tion. (a) ground-truth simulated fluorescence microscopy imaging data and (b) simulated measured 
fluorescence microscopy imaging data, (c)-(f) deconvolution results for b ((c); LR, (d): MCMC- 
WH, (e): MAP-Hunt, (f): MAP-D. Contrast is significantly improved in MAP-D when compared to 
b, along with noticeable SNR increases. 
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Figure 2: Application to fluorescence microscopy data of biological specimens, using fluorescence 
microscopy imaging of ocular surface wash, (a) Cell aggregates can be observed, some staining with 
fluorescein (green) while others stain red with the dead stain (ethidium bromide), (f) live/dead (cal- 
cein blue/ethidium bromide-red) fluorescent image of two corneal epithelial cells. The left and right 
dotted white lines found in a mark the plots shown in k and 1, respectively. The top row corresponds 
to deconvolution results for a ((b); LR, (c); MCMC-WH, (d); MAP-Hunt, (e); MAP-D. Contrast 
and resolution is signihcantly improved in MAP-D when compared to a, along with noticeable SNR 
increases when compared to b-d. The bottom row corresponds to deconvolution results for f ((g); 
LR, (h); MCMC-WH, (i); MAP-Hunt, (j); MAP-D. Similarly, contrast and resolution is signihcantly 
improved in MAP-D when compared to f, along with noticeable SNR increases when compared to 
g-j. (k) Intensity-normalized line plot (red channel) through an area of interest in a. Contrast is 
signihcantly enhanced in MAP-D when compared to the original acquisition. (1) Intensity-normalized 
line plot (blue channel) through an background area in a. SNR is noticeably improved in MAP-D 
when compared the other tested methods. 
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Figure 3: Application to fluorescence microscopy data of biological specimens, (a) fluorescence mi¬ 
croscopy imaging data of an ocular surface wash following sodium fluorescein instillation. Debris 
and dead corneal epithelial cells can be observed. The deconvolution results for a ((b); LR, (c); 
MCMC-WH, (d): MAP-Hunt, (e); MAP-D). Contrast and resolution is signihcantly improved in 
MAP-D when compared to a, along with noticeable SNR increases when compared to b-d. (f) and 
(k) fluorescence microscopy imaging data of an ocular surface wash following sodium fluorescein in¬ 
stillation. A potential fluorescein-stained cell (green staining) surrounded by dead corneal epithelial 
cells are somewhat visible. The deconvolution results for f,k ((g,l): LR, (h,m): MCMC-WH, (i,n); 
MAP-Hunt, (j,o): MAP-D). Contrast and resolution is signihcantly improved in MAP-D when com¬ 
pared to f and k, along with noticeable SNR increases when compared to g-i and 1-n, respectively, 
(p) huorescence microscopy imaging data of an ocular surface wash following sodium huorescein 
instillation: three potential huorescein-stained cell (green intra-cellular staining) and each nucleus 
stained with ethiudium bromide (red) indicating cell death. The deconvolution results for p ((q); 
LR, (r); MCMC-WH, (s); MAP-Hunt, (t); MAP-D). The contrast of the cell wall is improved in 
MAP-D compared to p with SNR increases compared to q-s. (u) imaging apparatus for obtaining 
huorescence microscopy acquisitions of cells obtained from an ocular surface wash following sodium 
huorescein instillation. 
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